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Abstract 



This study used hierarchical linear regression to examine the effects of taking specific 
high school courses on students’ ACT performance in English, Mathematics, and Science. In 
addition, hierarchical logistic regression was used to examine the benefits of taking specific 
courses on students’ likelihood of meeting or exceeding the ACT College Readiness Benchmarks 
in these three subject areas. All effects were examined in the context of the high schools students 
attended. Moreover, school characteristics were examined to determine whether course work 
effects were differentially influenced by the characteristics of the schools students attended. 

The data for the study consisted of 403,381 students from 10,792 high schools who took 
PLAN as sophomores and the ACT as juniors or seniors. Patterns of course work taken were 
developed from the Course Grade Information Section of the ACT. 

The results supported the use of hierarchical modeling for examining the contribution of 
course work taken to ACT performance. The relationships between course work taken and ACT 
performance were influenced by the characteristics of schools, most often the accrediting region 
of the school. 
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Using Hierarchical Modeling to Examine Course Work and ACT Score Relationships 

Across High Schools^ 

Hierarchical regression modeling is becoming increasingly popular as a methodology for 
analyzing data in the social sciences. It is used in situations where data are nested within groups, 
when the outcome variable of interest may depend on both individual characteristics and group 
membership. For example, students differ in the high school courses they take and their overall 
achievement, but there are also differences across schools in the types of courses they offer and 
their grading standards in those courses. Failing to account for both sources of variability in a 
regression model can result in incorrect conclusions about predictor-outcome relationships 
(Snidjers & Bosker, 1999). 

Basic Structure of Hierarchical Modeling 

Hierarchical models are extensions of regression models, such as basic linear and logistic 
regression models. For example, a researcher is interested in predicting high school GPA at 
graduation from students’ gender and the number of years of mathematics course work taken in 
high school. High school GPA and the predictors are student-level variables. A basic linear 
regression model is appropriate, as in; 

Yi = Bo + BiXii + B2Xi2 + Ri 

where high school GPA (H) is predicted from gender {Xu) and mathematics course work {Xu). 
The weights Bo, Bj, and B 2 are regression coefficients; Ri is the error term and is assumed to be 
independent across students. However, the effects of course work and gender on high school 
GPA may depend on the high school students attend. 



’ Results from this research are also summarized in the ACT Policy Report Courses Count: Preparing Students for 
Postsecondary Success, 2005. 
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Hierarchical modeling allows researchers to address issues of this type by partitioning 
variability across, as in this example, students and schools. It simultaneously fits two sets of 
regression equations: one at level 1 (the student level) and a second set at level 2 (the school 
level). Researchers can then investigate the extent to which, for example, class size might 
influence the relationship across schools between mathematics course work and high school 
GPA. 

The level 1 regression equation is analogous to the standard multiple linear regression 
equation shown above, but where Y is the outcome variable (high school GPA) for students 
within schools and the regression coefficients (J^) are specific to each school j: 

^ij + A 2jX2ij + Rij 

The level 2, or school-level, models consist of regression equations for each of the level 1 
regression coefficients, as in 

Poj = yoo + yoi Zj + Uoj 
Pij = yio + yn Zj + Uij 
Pij = yio + y 2 i Zj + Uy 

where z represents a group-level covariate, such as class size, yoo represents the average group 
mean (i.e., intercept) across groups when z = 0, yio and yio represent the average slopes for level 
1 variables xi and X 2 across groups when z = 0, and yoi, yn, and y 2 i represent the mean differences 
in intercepts and slopes associated with z. A group-level covariate may also be centered around 
its mean, in which case the interpretations of level 2 slopes and intercepts are not restricted to the 
condition that z = 0. 

In most hierarchical models, the level 2 equations include random effects that are 
associated with different groups {Uj). Random level 2 effects reflect the fact that the level 2 
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models are an approximation. For groups with small sample sizes, ineluding random effeets 
avoids overfitting of the model and obtaining large standard errors. Moreover, random level 2 
effeets enable researehers to test for the effeets of group-level predietors, as in the example, 
where elass size was used to prediet the level 1 eoeffieients. In addition, random effeets allow 
researehers to generalize their findings to the broader population of groups, rather than only 
applying their eonelusions to the groups ineluded in the study. For a eomplete discussion of 
hierarchical modeling, see Raudenbush and Bryk (2002) or Snidjers and Bosker (1999). 

ACT Research on Course Work and Achievement 

ACT research has shown that taking rigorous, college preparatory mathematics courses is 
associated with higher ACT Mathematics and Composite scores, (e.g., ACT, 2004; Noble, 
Davenport, and Sawyer, 2001; Noble, Roberts, and Sawyer, 2006). Schiel, Pommerich, and 
Noble (1996) statistically controlled for prior achievement using PLAN scores, and found 
substantive increases in average ACT Mathematics and Science scores associated with taking 
upper-level mathematics and science courses. 

The majority of research over the past 10 years on ACT scores and high school course 
work has statistically controlled for high school attended (e.g.. Noble, Roberts, and Sawyer, 
2006; Schiel, Pommerich, and Noble, 1996). In some studies, effect-coded dummy variables 
were used to represent high school attended in traditional multiple linear regression models (e.g.. 
Noble, Davenport, Schiel, and Pommerich, 1999). In other studies (e.g, Schiel, Pommerich, and 
Noble, 1996), within-high school regression models were developed and then the distributions of 
regression coefficients were summarized across schools using median, minimum, and maximum 
values. In all cases, high school attended had a significant effect on the contribution of course 
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work taken, either through redueing the amount of unexplained varianee in the eriterion variable 
or illustrating the variability of the eourse work regression eoeffieients aeross high sehools. 

In effeet-eoded dummy variable models, as in the Noble et al. (1999) study, eaeh dummy 
variable represents the differenee between a given high sehool’s performanee and the average 
performance of all high schools on the criterion variable (i.e., an adjustment to the intercept 
term). Unless dummy variable by course work interaction terms are included in the model, the 
course work regression coefficients are forced to be equal across high schools. With a large 
number of schools, such interaction terms become very complex and the model difficult to 
interpret. Moreover, regardless of the interaction terms, using dummy variables can result in 
inflated R values, due to the number of variables in the model; an increased Type 1 error rate 
when evaluating school-level differences; and an increased Type 2 error rate when evaluating the 
effects of student-level differences within schools (Snidjers and Bosker, 1999). In comparison, 
hierarchical models can include school-level intercept and slope adjustments for all regression 
coefficients in the model, and minimize R inflation and Type 1 and Type 2 error rates. 

Using within-school regression models, as in the Schiel et al. (1996) study, captures 
variability across high schools in both the slope and intercept terms. However, this approach 
suffers from two important limitations; First, in order to develop within-school regression 
models, there must be a sufficient number of students within each school to develop stable 
results. Very small schools must be ignored, resulting in a possible biasing of the results. 
Second, though course work variables were allowed to vary across schools, this approach fails to 
quantify variability in the regression coefficients across schools; only within-school variability is 
included in the error terms for each regression coefficient. In contrast, hierarchical modeling 
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allows for very small sample sizes and applies the appropriate error terms for sehool-level and 
student-level effects. 

Given the potential benefits of hierarchical modeling over other methodologies used 
previously, in this study we used hierarchical regression to examine the effects of taking specific 
high school courses on student’s ACT performance in English, Mathematics, and Science. All 
effects were examined in the context of the high schools students attended. Moreover, school 
characteristics were examined to determine whether course work effects were differentially 
influenced by the characteristics of the schools students attended. 

ACT conducted a study in spring 2003 (Allen & Sconing, 2005) to identify ACT scores 
associated with successful performance in first year college courses. The results identified those 
scores corresponding to a 50% chance of a B or higher, or at least a 75% chance of a C or higher, 
in English Composition, College Algebra, and College Biology. These scores included an ACT 
English score of 18, an ACT Mathematics score of 22, and an ACT Science score of 24. Thus, 
ACT-tested students who meet or exceed these Benchmarks are likely to be successful in entry- 
level, credit-bearing college courses. However, to-date no research has been done examining the 
relationship between high school course work taken and meeting or exceeding these 
Benchmarks. Eor this study we therefore examined the effects of taking specific courses on 
meeting ACT’s College Readiness Benchmarks. 

Data 

The ACT and PEAN tests (ACT, 1997; 1999) are intended to measure high-order 
thinking skills in four content areas (English, Mathematics, Reading, and Science). Scores range 
from 1 to 32 for PEAN and 1 to 36 for the ACT. The Composite score is the rounded arithmetic 
average of the four subject area scores. 
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In general, students who are high aehieving at the time they enter high school take more 
rigorous, higher-level courses than do students who are lower-achieving. Therefore, it is 
important to differentiate whether students receive higher ACT scores because of the courses 
they take or because of their prior achievement. For this study, prior achievement was 
statistically controlled by including students’ PLAN subject area scores in the hierarchical 
models. 

The PLAN/ACT cohort file for the 2003 graduating class contained matched records of 
students who completed PLAN during their sophomore year (2000-01) and the ACT during their 
junior or senior year, prior to graduating in 2003 . If students took the ACT more than once, only 
the most recent ACT record was used. Each record included PLAN and ACT scores (in English, 
Mathematics, and Science), race/ethnicity, grade level at the time of taking the ACT, and self- 
reported course work information. Three dichotomous variables (1,0) were also created for each 
student, representing whether or not s/he met each of the three College Readiness Benchmarks. 

Students’ grade level at the time of ACT testing was also included in the hierarchical 
models. The course work students report having taken by the time they complete the ACT 
depends on their grade level. In addition, the PEAN test is typically taken by high school 
sophomores; students can take the ACT as juniors and/or as seniors. The time span between 
PLAN and ACT testing can therefore range from a few months to two or more years. 

The Course Grade Information Section (CGIS) of the ACT provides information about 
students’ course work in 30 specific high school courses. Students are asked to indicate whether 
they have taken or are currently taking a particular course, or whether they plan to take it in the 



^ Virtually every student in the matehed file attended the same high sehool at the time of ACT testing that he or she 
had attended at the time of PLAN testing. There were, however, some students who transferred to different sehools 
prior to taking the ACT 
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future. In earlier studies, students were found to report these data with a high degree of aeouraey 
relative to information provided in their transeripts (Sawyer, Laing and Houston, 1988; Valiga, 
1987). Dummy variables were ereated to make speeifie eourse eomparisons; the eourse patterns 
were based on previous researeh (ACT, 2004; Noble, Davenport, Sehiel, & Pommerieh, 1999) 
and were eonstrueted sueh that the ineremental benefit of speeifie eourses eould be determined. 
The patterns studied ineluded the following: 

1. English 9, English 10, and English 1 1 vs. taking less than these three eourses 

2. English 9, English 10, and English 11 vs. English 9, English 10, and English 11 and 
one or more foreign languages 

3. Algebra 1, Algebra 2, and Geometry vs. taking less than these three eourses 

4. Algebra 1, Algebra 2, and Geometry vs. Algebra 1, Algebra 2, Geometry and Other 
Advaneed Mathematies 

5. Algebra 1, Algebra 2, and Geometry vs. Algebra 1, Algebra 2, Geometry and 
Trigonometry 

6. Algebra 1, Algebra 2, and Geometry vs. Algebra 1, Algebra 2, Geometry, 
Trigonometry, and Caleulus 

7. Algebra 1, Algebra 2, and Geometry vs. Algebra 1, Algebra 2, Geometry, Other 
Advaneed Math, Trigonometry, and Caleulus 

8. General Seienee and Biology, or Biology only (denoted as Biology), vs. taking 
General Seienee 

9. Biology vs. Biology and Chemistry 

10. Biology vs. Biology, Chemistry, and Physies 
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College English instructors rank grammar high in importance as a prerequisite skill for 
college English Composition (ACT, 2003). Therefore, the ACT English test includes basic 
grammar and usage in its test specihcations. Research shows that taking a foreign language 
requires some understanding of rules of grammar in order to understand the grammatical system 
specific to that language (ZhonggangGao, 2001). Taking one or more foreign languages was 
therefore included as a variable to determine whether it was a viable predictor of ACT English 
score and the ACT English Benchmark, over and above taking English 9-11. 

Characteristics of the schools in the study were obtained from the Market Data Retrieval 
(MDR) data history and were matched to each student record. These variables were selected 
based on prior research (e.g.. Noble, 2003; Roberts and Noble, 2004; Schiel, Pommerich, and 
Noble, 1996) and included the following: 

1 . School type (public or private) 

2. Class size (in hundreds) 

3. Eocation (urban, rural, suburban) 

4. Accrediting region (South, Northwest, North Central, West, Northeast) 

5. School district type (single or multi-school district) 

Dummy variables were created for school type, location, and school district type. Eocation was 
coded such that rural and suburban schools were compared to urban schools. Urban schools 
were selected as the reference group primarily because they tend to have larger achievement 
gains over time than rural schools and smaller achievement gains over time than suburban 
schools (Roberts & Noble, 2004; Schiel, Pommerich, & Noble, 1996). Effect-coded dummy 
variables were used for accrediting region so that each region could be compared to the total 
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group; no one region was of interest as a referenee group. ACT high school codes were used to 
identify each high school. 

The total sample consisted of 403,381 students from 10,792 high schools. Each student 
was required to be either a junior or a senior, to have valid PLAN and ACT Composite scores, 
and had to have valid values for all school characteristic variables. 

Method 

Means and standard deviations were calculated on all student- and school-level 
continuous variables. Percentages were calculated for all categorical variables. 

Hierarchical linear regression was used to model ACT English, Mathematics, and 
Science scores. Similarly, hierarchical logistic regression was used to model the probability of 
students’ meeting or exceeding the ACT English, Mathematics, and Science College Readiness 
Benchmarks. The student-level (level 1) independent variables included the relevant PLAN 
score, student grade level (11 or 12), and specific course pattern indicators. All analyses were 
conducted using HEM version 5 (Raudenbush, Bryk, Cheong, and Congdon, 2000). 

Two sets of models were developed: The first set of models treated student-level 
regression coefficients as random effects across high schools, but excluded high school 
characteristics as predictors of these coefficients. The second set also treated student-level 
regression coefficients as random effects across high schools, but also included school 
characteristics as predictors of the level 1 intercepts and slopes. All nonsignificant (p > .01) 
school characteristic variables were dropped from each model. 

The linear regression results were evaluated in terms of the change in average ACT 
scores associated with taking a particular course or course pattern, and the effects of school 
characteristics on this change. A linear regression coefficient for a particular course (i.e., a 
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student-level variable) can be interpreted as the average increase in ACT score associated with 
taking the course. Similarly, the school-level variables were coded such that the regression 
coefficient associated with a certain school characteristic could be interpreted as the typical 
increase in the average student-level coefficient that was associated with that school 
characteristic. The level 2 intercept (y/o) was the student-level regression coefficient averaged 
across all schools. 

The logistic regression results were evaluated in terms of the proportional increase in the 
odds of meeting or exceeding a particular Benchmark associated with taking a particular course 
pattern (odds ratio). The odds ratio is the ratio of the odds for the two groups defined by their 
taking or not taking a particular course or course pattern. 

odds ratio 

p(0)/[l-/7(0)] 

where />(0) = probability of not meeting/exceeding the Benchmark (0), 
given that a particular course pattern was not taken 
p{\) = probability of meeting/exceeding the Benchmark (1), given 
that a particular course pattern was taken 
ji = the regression coefficient associated with taking a 
particular course or course pattern (i.e., estimated log odds 
of meeting/exceeding the Benchmark), given the other 
variables in the model. 

Coefficients for the level 2 school characteristics were similarly interpreted; They 
reflected the typical adjustment in the average level 1 regression coefficient (across schools) that 



was associated with a school characteristic. 
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Results 

Descriptive Statistics 

Descriptive statistics for all variables used in the study are shown in Tables 1 and 2. 
Means and standard deviations, or percentages, are shown for both student-level and school-level 
variables. 

PLAN and ACT mean scores (see Table 1 on following page) were generally higher than 
those of students nationally (ACT, 2000; 2003). As a result, a higher percentage of students met 
or exceeded the College Readiness Benchmarks than did students nationally. The majority of 
students were high school seniors (57%) and Caucasian American (76%). 

Almost all students had taken English 9-11, or English 9-11 and one or more foreign 
languages by the time they took the ACT. Thirty-four percent had taken only Algebra 1, Algebra 
2, and Geometry, while 50% had taken additional mathematics course work beyond these three 
courses. In science, 38% had taken only Biology, but 33% had taken Biology and Chemistry. 
Only 26% had taken Biology, Chemistry, and Physics. 
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TABLE 1 

Descriptive Statistics for All Student-Level Variables 



Variable 


Mean 


SD 


Percentage 


Test scores 


PLAN English 


18.8 


4.46 




PLAN Mathematics 


19.0 


4.29 




PLAN Science 


19.3 


3.07 




ACT English 


21.3 


5.76 




ACT Mathematics 


21.2 


5.15 




ACT Science 


21.5 


4.62 




Meets/exceeds English Benchmark 






74 


Meets/exceeds Mathematics Benchmark 






44 


Meets/exceeds Science Benchmark 






30 


Demographic characteristic 


12^*^ grade student 






57 


African American 






9 


Caucasian American 






76 


Hispanic 






5 


Asian American 






3 


Other 






4 


Course work taken 


Less than English 9-11 






1 


English 9-11 






6 


English 9-1 1 & one or more foreign languages 






94 


Less than Alg. 1, Alg. 2, & Geometry 






12 


Alg. 1, Alg. 2, & Geometry 






34 


Beyond Alg. 1, Alg. 2, & Geometry: 








Other Adv Math only 






13 


Trigonometry only 






17 


Trig & Other Adv. Math only 






11 


Trig & Calculus only 






4 


Other Adv Math, Trig & Calculus 






5 


General Science 






2 


Biology 






38 


Beyond Biology: 








Chemistry 






33 


Chemistry & Physics 






26 
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As shown in Table 2, the large majority of sehools were publie sehools (82%). Slightly 
more than half (54%) were rural sehools and from the North Central aeerediting region. Less 
than 40% (38%) were multi-high sehool distriets. The average elass size was 239 students. 

TABLE 2 

Descriptive Statistics for All School-Level Variables 



Variable 


Mean (SD)/ 
percentage 


Class size (in hundreds) 


2.39 (2.61) 


Sehool type 




Publie sehool 


82 


Private 


18 


Urbanieity 




Rural 


54 


Suburban 


26 


Urban 


20 


Region 




South 


30 


Northwest 


5 


North Central 


56 


West 


4 


Northeast 


5 


Type of distriet 




Multi-high sehool 


38 


Single high sehool 


62 



Modeling ACT Scores: Hierarchical Linear Regression Results 

The hierarehieal linear regression results exeluding sehool oharaeteristies are shown in 
Tables 3a-3e. Eaeh table ineludes the unstandardized regression eoeffieients for eaeh variable; all 
regression eoeffieients were statistieally signifieant (p < .01) unless otherwise noted. Aeross all 
models exeept English 9-11 vs. less than English 9-11, varianees aeross high sehools for the 
intereept and slopes (i.e., random effeets) were statistieally signifieant (p < .01). These results 
supported the use of hierarehieal models with random effeets for this study. It should be noted, 
however, that the varianee aeross high sehools for the PEAN seore slope, though statistieally 
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significant, was small (.01 -.03 across all models) relative to the magnitude of the slope (.75 to 
.99). 



As shown in Table 3a, two sets of eourse work variables were used to model ACT 
English seore; One model ineluded taking English 9-11 (Model 1) and the other included taking 
both English 9-11 and one or more foreign languages (Model 2). Eor both models, PEAN 
English seore and grade level were positively related to ACT English seore. A one unit inerease 
in PEAN English seore eorresponded to about a one unit inerease in ACT English seore. 
Moreover, high sehool seniors, on average, seored about .30 to .32 units higher on ACT English 
than did juniors. Overall, the models explained 58% and 60% of the varianee (R ) in students’ 
ACT English seores. 



TABLE 3a 

Hierarchical Linear Regression Coefficients for ACT English Score 



Model 


Course work comparison 


Regression coefficient 


Level 1 
R^ 


Intercept 


PLAN 

English 


Grade 

level 


Course 

work 


1 


English 9-11 vs. less than 
these 3 eourses 


2.20 


.98 


.30 


-.72 


.58 


2 


English 9-1 1 & 1 or more 
foreign languages vs. 
English 9-11 


1.33 


.99 


.32 


1.12 


.60 



Taking English 9-11 (Model 1) was assoeiated with a .72 deerease in average ACT 
English score, eompared to taking fewer eourses, given students’ prior aehievement and grade 
level at the time of taking the ACT. This deerease is likely due to the faet that very few students 
reported taking less than English 9-11 (1%). Sinee English 9-12 is typieally required of all high 
sehool graduates, it is likely that these students did not report their aetual English eourse work 
taken, or were suffieiently advaneed that they were not required to take all three eourses in high 
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school. In contrast, taking one or more foreign languages over and above English 9-11 (Model 2) 
increased students’ ACT English seore, on average, by 1.12 score units, eompared to taking only 
English 9- IE 

Table 3b contains the results for ACT Mathematics score. Six models were developed, 
eaeh with a unique eourse work variable. Models 2-6 included students taking Algebra 1, 
Algebra 2, and Geometry as the eomparison group to evaluate the eontribution of additional 
eourses. 



TABLE 3b 

Hierarchical Linear Regression Coefficients for ACT Mathematics Score 



Model 


Course work comparison 


Regression coefficient 


Level 1 
R^ 


Intercept 


PLAN 

Math 


Grade 

level 


Course 

work 


1 


Alg 1, Alg 2, and Geometry vs. less 
these eourses 


5.03 


.75 


-.45 


E07 


.52 


Alg 1, Alg 2, and Geometry vs. 


2 


Alg 1, Alg 2, Geometry, & 
Other Adv Math only 


5.65 


.79 


-.66 


EOl 


.52 


3 


Alg 1, Alg 2, Geometry, & Trig 
only 


5.63 


.79 


-.70 


1.52 


.59 


4 


Alg 1, Alg 2, Geometry, Trig & 
Other Adv Math only 


5.91 


.78 


-.72 


2.02 


.60 


5 


Alg 1, Alg 2, Geometry, Trig & 
Calc only 


5.84 


.78 


-.62 


2.91 


.60 


6 


Alg 1, Alg 2, Geometry, Other 
Adv Math, Trig & Cale 


5.90 


.77 


-.62 


3.16 


.63 



Across all models, a one -unit increase in PLAN Mathematies score was assoeiated with 
an average ACT Mathematics score inerease of .75 to .79. On average, juniors outscored seniors 
by .45 to .72 score units. The predietor variables explained 52% to 63% of the varianee in 



students’ ACT Mathematies seores. 
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Taking Algebra 1, Algebra 2, and Geometry (Model 1) was associated with an average 
ACT Mathematics score increase of about 1 . 1 score units, compared with taking less than these 
three courses. Taking either Other Advanced Mathematics or Trigonometry (Model 2 or 3), in 
addition to these three courses, resulted in an average increase in ACT Mathematics score of 
1.01 to 1.52 score units. Taking Other Advanced Mathematics and Trigonometry, or 
Trigonometry and Calculus (Model 4 or 5), increased ACT Mathematics scores, on average, by 
more than 2.00 score units. The greatest average score increase from mathematics course work 
resulted from taking Other Advanced Mathematics, Trigonometry, and Calculus (Model 6), in 
addition to Algebra 1, Algebra 2, and Geometry (3.16 score units). 

Table 3c contains the results for ACT Science. Three models were developed, each with a 
unique course work variable. Models 2 and 3 included students taking Biology as the comparison 
group to evaluate the contribution of additional courses. 

TABLE 3c 

Hierarchical Linear Regression Coefficients for ACT Science Score 



Model 


Course work comparison 


Regression coefficient 


Level 1 
R^ 


Intercept 


PLAN 

Science 


Grade 

level 


Course 

work 


1 


Biology vs. General Science 


4.70 


.78 


-.07* 


.46 


.28 


Biology vs. 


2 


Biology & Chemistry 


4.26 


.83 


-.43 


1.29 


.37 


3 


Biology, Chemistry, & Physics 


4.23 


.84 


-.48 


2.41 


.47 



* p> .0 



A one unit increase in PLAN Science score was associated with an average ACT Science 
score increase of .78 to .84. Juniors typically outscored seniors, with ACT Science scores 
typically .07 to .48 score units higher than those for seniors. Between 28% and 47% of the 
variance in students’ ACT Science scores was explained by these models. 
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Compared with taking General Science only, taking General Science and Biology, or 
Biology alone (Model 1), resulted in an average ACT Science score increase of .46. In 
comparison, taking Biology and Chemistry, or Biology, Chemistry, and Physics (Model 2 or 3), 
was associated with an average ACT Science score increase of over 1 .2 score units, compared to 
taking only Biology. 

Differences in course work effects by school characteristic. The hierarchical linear 
regression models including school characteristics are provided in Appendix A. These tables 
include all statistically significant (p < .01) effects; non-significant results are not reported, 
except where multiple dummy variables are used to represent categories of one variable (e.g., 
region). Non-significant results are noted in these cases. The results including school 
characteristics showed that the contributions of the level 1 variables to ACT performance 
differed significantly across school characteristics. For almost all models, the PLAN score 
regression coefficient differed significantly by region and, for Mathematics, by school type. 
However, the magnitude of the coefficients for school type was quite small and the coefficients 
for individual regions were often not statistically significant. 

The regression coefficients for grade level showed that the relationship between ACT 
performance and grade level generally differed across regions. Typically, seniors outscored 
juniors in the Southern and North Central regions, but juniors outscored seniors in the 
Northeastern region. 

The intercept coefficients illustrate the differences in ACT score across high schools of 
various types. For example, the results for the first mathematics model showed that public 
schools, and those from the Southern region, had higher ACT Mathematics scores than those 
from private schools, or those from other accrediting regions. 
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For ACT English, the effect of taking English 9-1 1 did not differ by school characteristic 
(Model 1). However, the effect of taking English 9-11 and one or more foreign languages 
differed significantly by school type, location, and region (Model 2). Average score increases 
associated with taking these courses were greater for private schools, suburban schools, and 
schools in the Southern and North Central regions. 

With the exception of taking Algebra 1, Algebra 2, Geometry, and Other Advanced 
Mathematics only (Model 2), taking mathematics courses was associated with a larger average 
ACT Mathematics score increase for students from rural and suburban schools than for those 
from urban schools. Earger average score increases associated with taking Algebra 1, Algebra 2, 
and Geometry (Model 1); Other Advanced Mathematics, Trigonometry, and these three courses 
(Model 4); and Trigonometry, Calculus, and these three courses (Model 5) were also found for 
students from schools in the North Central accrediting region. Students from schools in the 
Southern accrediting region had smaller than average score increases associated with taking 
Algebra 1, Algebra 2, and Geometry (Model 1), but larger increases associated with taking 
Algebra 1, Algebra 2, Geometry, Trigonometry, and Calculus (Model 5). Students from schools 
from the Western accrediting region had much smaller than average ACT Mathematics score 
increases associated with taking Algebra 1, Algebra 2, Geometry, Other Advanced Mathematics, 
Trigonometry, and Calculus (Model 6). 

Taking Biology alone (Model 1), Biology and Chemistry (Model 2), or taking Physics in 
addition to these two courses (Model 3), was associated with larger average ACT Science score 
increases for students from rural and suburban schools, and students from schools in the North 
Central accrediting region (excluding Biology only), than for students from urban schools and 
those from the Southern, Northwestern, and Western accrediting regions. Taking Biology and 
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Chemistry, or these two eourses and Physios (Models 2 or 3), was also assooiated with larger 
than average ACT Scienoe score increases for students from the Northeastern accrediting region. 
Modeling ACT College Readiness Benchmarks: Hierarchical Logistic Regression Results 

The unstandardized logistic regression coefficients are reported in Appendix B1 for 
models excluding school characteristics. The odds ratios corresponding to particular course work 
variables are described below. 

For the ACT English Benchmark, logistic regression models could not be developed for 
English 9-11. Very few students took less than English 9-11 and failed to meet the Benchmark. 
Compared to taking only English 9-11, however, also taking one or more foreign languages was 
associated with a 97% increase in the odds of students meeting or exceeding the Benchmark. 

Eigure 1 illustrates the increases in the odds of meeting the ACT Mathematics 
Benchmark by taking Algebra 1, Algebra 2, and Geometry, as well as taking other courses over 
and above Algebra 1, Algebra 2, and Geometry. Compared to taking fewer courses, taking 
Algebra 1, Algebra 2, and Geometry was associated with an increase in the odds of students 
meeting the Benchmark by 148%. The odds of meeting the Benchmark were about 2% times 
greater for students taking these courses, compared to students taking fewer courses. 
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FIGURE 1. Increase in Odds of Meeting or Exeeeding ACT Mathematies Benehmark by Taking 

Speeifie Mathematies Course Work 



500 ^ 




Alg 1 , Alg 2, Alg 1 , Alg 2, Alg 1 , Alg 2, Alg 1 , Alg 2, Alg 1 , Alg 2, Alg 1 , Alg 2, 
& Geom Geom, & Geom, &Trig Geom, Other Geom, Trig, Geom, Other 
Other Adv Adv Math, & & Calc Adv Math, 

Math Trig Trig, & Calc 

Course work taken 



As was found earlier with the linear regression results, taking upper-level mathematies 
eourses beyond Algebra 1, Algebra 2, and Geometry was assoeiated with substantial inereases in 
the odds of students meeting or exeeeding the ACT Mathematies Benchmark (by 88% to 481%). 
In other words, the odds of students meeting the Mathematies Benehmark were 1.8 to about six 
times greater for students taking these courses, eompared to taking only Algebra 1, Algebra 2, 
and Geometry. 

Taking Biology and Chemistry, eompared to taking only Biology, was assoeiated with a 
103% inerease in the odds of meeting or exeeeding the ACT Seienee Benchmark. Compared to 
taking Biology alone, taking Biology, Chemistry, and Physies inereased the odds by 271%. 
Logistie models eould not be developed for Biology alone; too few students taking Biology alone 



or General Seienee met the ACT Seienee Benehmark. 
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Differences in course work effects by school characteristic. The hierarchical logistic 
regression models including school characteristics are provided in Appendix B2. These tables 
include all statistically significant (p < .01) effects; non-significant results are indicated with an 
asterisk (*). As noted above, logistic models could not be developed for Biology alone; too few 
students taking Biology alone or General Science met the ACT Science Benchmark. 

The logistic regression results differed somewhat from the linear regression results. 
However, school characteristics were still found to influence the relationships between the level 
1 predictors and ACT performance. In particular, the association between PLAN score and ACT 
performance still differed by region (and by school type for ACT Mathematics), though the 
relationship remained small. 

Taking English 9-11 and one or more foreign languages (Model 2) was associated with a 
greater increase in the odds of meeting the ACT English Benchmark for students from private 
high schools and single school districts. Regional differences were found for this course 
sequence, with students from Southern schools who took these courses having greater increases 
in the odds of meeting the ACT English Benchmark than students from other regions. 

Attending a public school was associated with greater increases in odds of meeting the 
ACT Mathematics Benchmark by taking Other Advanced Mathematics, or Other Advanced 
Mathematics and Trigonometry (Model 2 or 4), compared to students attending private schools. 
In contrast, attending a private school was associated with greater odds by taking all six 
mathematics courses (Model 6). Eor most of the mathematics course patterns, attending a rural or 
suburban school was associated with greater odds of achieving the Benchmark than attending an 
urban school. Regional differences were found for only two mathematics course patterns: 
Algebra 1, Algebra 2, and Geometry (Model 1), where these courses were associated with higher 
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increases in odds for students from the North Central region, and smaller increases in odds for 
students from the Southern region, and taking all six courses (Model 6), where greater inereases 
in odds were found for students from sehools in the Southern region. 

For Science, greater increases in the odds of achieving the Benchmark associated with 
taking Biology, Chemistry, and Physies (Model 3) were found for students from rural and 
suburban schools and those from the North Central and Northeastern accrediting regions, 
eompared to students from urban schools or those from other regions. Students from sehools in 
the Northeastern region had similar increases in odds associated with taking Biology and 
Chemistry only (Model 2). In contrast, taking either set of courses was associated with smaller 
increases in the odds of aehieving the Benehmark for students from the Southern aeerediting 
region. Students who took Biology and Chemistry (Model 2) and were from private schools also 
had smaller than average increases in the odds of achieving the Benchmark. 

Conclusions 

The results of this study supported the use of hierarehical modeling for examining 
relationships between high school course work taken and ACT performance. The benefits of 
eourse work taken in high school for ACT performance depended on the high sehool students 
attended, regardless of prior achievement and grade level at testing. We found significant 
variability across high schools in nearly all regression eoefficients, thereby supporting the use of 
hierarchical modeling to address this issue. 

Compared to earlier studies, these models explained similar proportions of student-level 
variance in ACT scores. The magnitude of the regression eoefficients are difficult to compare, 
due to the other variables ineluded in the models and the manner in whieh the course work 



variables were coded. However, the overall trends were similar: higher level mathematics and 
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science courses corresponded to greater average increases in ACT Mathematies and Seienee 
scores, eompared to lower-level courses. 

The relationships between eourse work taken and ACT performance were influenced by 
the characteristics of schools, most often the accrediting region of the school. These results 
oeeurred for virtually all eourse work variables studied. In addition, overall sehool means 
(intereept terms) differed aeross high sehools and high school characteristics, thus arguing for 
hierarchical models that inelude level 2 equations for both the intereept and slope terms. 

Because the course work variables and models from earlier researeh differed from those 
used for this study, it is not possible to make speeific eomparisons between the results. However, 
the overall findings here are eonsistent with other ACT studies where high school attended was 
used explieitly in the models. Unlike other studies, however, statistieal signifieance tests eould 
be used directly on high school effects to evaluate the implications of specific high school 
characteristics on slopes and intereept terms. 

Implications 

Hierarchical modeling is a complex, but versatile approach for evaluating relationships 
between and among individual- and group-level variables. Single-level models oversimplify the 
eomplex nature of sueh relationships, failing to take into aeeount the numerous faetors that affeet 
edueational outeomes. With hierarchieal modeling, such issues can be addressed with a greater 
likelihood of finding real differences that are attributable to real events, and not to errors in the 



models themselves. 
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Appendix A 

Hierarchical Linear Models Including School Characteristics 



ACT English 







Level 2 regression coefficient 












Location 


Accred. region 


Multi- 








Pub vs. 


Class 


Rural vs. 


Suburban 












school 


Model 


Level 1 


Intercept 


priv. 


size 


urban 


vs. urban 


South 


NW 


NC 


W 


NE 


dist. 


1 


Intercept 


2.52 


-.40 




.04* 


.28 
















PLAN English 


.97 




<.01 






-.04 


.03 


>-.01* 


.02* 


<-.01* 






Grade level 


.35 
























Eng 9-11 vs. less than 
Eng. 9-11 


-.69 






















2 


Intercept 


.79 




.11 






-.36 


.29* 


.23* 


-.06* 


.28 


.28 




PLAN English 


1.01 










-.02 


.01* 


-.02 


.02* 


-.02 


-.02 




Grade level 


.33 




-.04 


.18 


.07* 


.12 


-.01* 


.04* 


.19* 


-.34 






Eng 9-1 1 & 1 or more 
foreign lang vs. Eng 9-11 


1.29 


-.59 




-.06* 


.26 


.54 


-.27* 


.29 


-.64* 


.07* 





*p>.0 






ACT Mathematics 







Level 2 regression coefficient 












Location 


Accred. region 


Multi- 








Pub vs. 


Class 


Rural vs. 


Suburban 












school 


Model 


Level 1 


Intercept 


priv. 


size 


urban 


vs. urban 


South 


NW 


NC 


W 


NE 


dist. 


1 


Intercept 


3.94 


.90 








.73 


-.34* 


.19* 


-.20* 


-.38* 






PLAN Mathematics 


.87 


-.10 


.01 






-.06 


.02* 


-.04 


.04* 


.05* 


-.01 




Grade level 


-.62 










.13* 


-.07* 


.14 


.16* 


-.37 


.10 




Alg 1, Alg 2, & Geom vs. 
less than these 3 courses 


.69 






.16 


.29 


-.24 


.19* 


.40 


-.10* 


-.25* 




Alg. 1, Alg. 2, & Geom. Vs 


2 


Intercept 


5.36 




.09 




















PLAN Mathematics 


.84 


-.05 




<.01* 


.02 


-.03 


.01* 


-.01* 


.02 


.01* 


-.01 




Grade level 


-.92 






.14 


.03* 


.16 


-.04* 


.18 


.07* 


-.38 


.10 




Alg 1, Alg 2, Geom, Other 
Adv Math 


.89 






.22 


.09* 














3 


Intercept 


5.38 




.08 




















PLAN Mathematics 


.84 


-.04 








-.03 


.01* 


>-.01 


.02 


.01 






Grade level 


-.85 










.25 


-.07* 


.16 


.07* 


-.42 






Alg 1, Alg 2, Geom, & 
Trig 


1.16 




.05 


.33 


.34 












-.14 


4 


Intercept 


5.84 




.08 






-.66 


.04* 


-.09* 


.39 


.32 






PLAN Mathematics 


.81 


-.04 






















Grade level 


-1.03 






.16 


.15 


.35 


.03* 


.19 


.11* 


-.67 






Alg 1, Alg 2, Geom, Trig 
& Other Adv Math 


1.64 






.35 


.36 


-.14* 


-.02 


.16 


-.11* 


.10* 




5 


Intercept 


5.55 




.08 




















PLAN Mathematics 


.82 


-.04 








-.03 


<.01* 


>-.01 


.02 


.01* 






Grade level 


-.73 










.20 


-.02* 


.12* 


.14* 


-.44 






Alg 1, Alg 2, Geom, Trig 
& Calculus 


2.18 




.05 


.46 


.38 


.22 


-.12* 


.21 


-.33* 


.02* 




6 


Intercept 


5.60 




.08 




















PLAN Mathematics 


.82 


-.04 








-.03 


<.01* 


>-.01 


.02 


.01* 






Grade level 


-.73 










.21 


-.02* 


.12* 


.08 


-.38 






Alg 1, Alg 2, Geom, Other 
Adv Math, Trig, & Calc 


2.80 






.33 


.35 


.12* 


.17 


.05* 


-.48 


.15* 





*p>.01 












ACT Science 







Level 2 regression coefficient 












Location 


Accred. region 


Multi- 








Pub vs. 


Class 


Rural vs. 


Suburban 












school 


Model 


Level 1 


Intercept 


priv. 


size 


urban 


vs. urban 


South 


NW 


NC 


W 


NE 


dist. 


1 


Intercept 


5.08 


-.32 






















PLAN Science 


.78 










-.03 


.04 


-.01 


.01* 


-.01* 






Grade level 


-.33 










.16* 


-.11* 


.35 


.02* 


-.42* 






Biology 


.21* 




.04 


.21 


.33 












-.21 


Biology vs. 


2 


Intercept 


4.51 


-.38 


.05 


.12* 


.24 












-.36 




PLAN Science 


.84 










-.03 


.04 


-.01 


.01* 


-.01* 






Grade level 


-.75 


.17 


-.02 






.17 


-.02* 


.24 


.03* 


-.43 


.12 




Biology & Chemistry 


1.18 


-.28 


.04 


.23 


.21 


-.17 


-.12* 


.16 


-.22* 


.35 




3 


Intercept 


3.77 


-.17 




.93 


.96 
















PLAN Science 


.87 






-.04 


-.04 


-.03 


.03 


-.01 


.01* 


-.01* 






Grade level 


-.57 




-.03 






.28 


.06* 


.25 


.03* 


-.62 






Biology, Chemistry, & 
Physics 


2.02 


-.41 


.09 


.56 


.36 


-.23 


-.15* 


.25 


-.51* 


.64 





*p>.01 





Appendix B1 



Hierarchical Logistic Models Excluding School Characteristics 
ACT English College Readiness Benchmark 



Model 


Course work comparison 


Regression coefficient 


Intercept 


PLAN 

English 


Grade 

level 


Course 

work 

(odds ratio) 


1 


English 9-11 vs. less than these 3 
courses 


** 


** 


** 


** 


2 


English 9- 1 1 & 1 or more foreign 
languages vs. English 9-11 


-8.04 


.49 


.02* 


.68 (1.97) 



** Insufficient numbers of students taking less than English 9-11 and not meeting the English Benchmark to 
calculate. 



ACT Mathematics College Readiness Benchmark 



Model 


Course work comparison 


Regression coefficient 


Intercept 


PLAN 

Math 


Grade 

level 


Course 

work 

(odds ratio) 


1 


Alg 1, Alg 2, and Geometry vs. less 
than these courses 


-10.29 


.47 


-.37 


.91 (2.48) 


Alg 1, Alg 2, and Geometry vs. 


2 


Alg 1 , Alg 2, Geometry, & Other 
Adv Math only 


-9.18 


.46 


-.40 


.63 (1.88) 


3 


Alg 1, Alg 2, Geometry, & Trig 
only 


-8.91 


.44 


-.43 


.90 (2.46) 


4 


Alg 1, Alg 2, Geometry, Trig & 
Other Adv Math only 


-8.86 


.44 


-.42 


1.15(3.16) 


5 


Alg 1, Alg 2, Geometry, Trig & 
Calc only 


-9.01 


.45 


-.40 


1.66 (5.26) 


6 


Alg 1 , Alg 2, Geometry, Other 
Adv Math, Trig & Calc 


-8.96 


.44 


-.40 


1.76 (5.81) 



ACT Science College Readiness Benchmark 



Model 


Course work comparison 


Regression coefficients 


Intercept 


PLAN 

Science 


Grade 

level 


Course 

work 

(odds ratio) 


1 


Biology vs. General Science 


** 


** 


** 




Biology vs. 


2 


Biology & Chemistry 


-10.97 


.48 


-.29 


.71 (2.03) 


3 


Biology, Chemistry, & Physics 


-10.24 


.44 


-.30 


1.31 (3.71) 



** Insufficient numbers of students meeting the Science Benchmark to calculate. 







Appendix B2 

Hierarchical Logistic Models Including School Characteristics 



ACT English College Readiness Benchmark 







Level 2 regression coefficient 












Location 


Accred. region 


Multi- 








Pub vs. 


Class 


Rural vs. 


Suburban 












school 


Model 


Level 1 


Intercept 


priv. 


size 


urban 


vs. urban 


South 


NW 


NC 


W 


NE 


dist. 


1 


Intercept 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 




PLAN English 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 




Grade level 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 




Eng 9-11 vs. less than 
Eng. 9-11 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


2 


Intercept 


-8.76 




.07 






.28* 


-.07* 


.48 


-.67* 


-.02* 






PLAN English 


.54 


-.01 








-.04 


.03 


-.03 


.05 


-.01* 






Grade level 


.11 




-.03 


.08 


.02* 


<.01* 


-.02* 


-.03* 


.27 


-.22 






Eng 9-1 1 & 1 or more 
foreign lang vs. Eng 9-11 


.69 


-.18 




>-.01* 


.18 


.31 


-.19* 


.10* 


-.42* 


.21* 


-.07 



* p > .01 

** Insufficient numbers of students taking less than English 9-11 and not meeting the English Benchmark to calculate. 




ACT Mathematics College Readiness Benchmark 







Level 2 regression coefficient 












Location 


Accred. region 


Multi- 








Pub vs. 


Class 


Rural vs. 


Suburban 












school 


Model 


Level 1 


Intercept 


priv. 


size 


urban 


vs. urban 


South 


NW 


NC 


W 


NE 


dist. 


1 


Intercept 


-10.61 




.06 


-.38 


.22* 












-.15 




PLAN Mathematics 


.52 


-.03 




.02 


-.01* 


>-.01* 


>-.01* 


-.02 


.01* 


.01 






Grade level 


-.41 




















.09 




Alg 1, Alg 2, & Geom vs. 
less than these 3 courses 


.68 










-.26 


-.01* 


.40 


.13* 


-.26* 




Alg 1, Alg 2, & Geom. vs. 


2 


Intercept 


-9.99 




.14 






-.22* 


.18* 


.49 


.92 


-1.38 


-.43 




PLAN Mathematics 


.53 


-.03 


>-.01 






-.01* 


-.01* 


-.03 


-.03* 


.08 


.02 




Grade level 


-.52 






.12 


.11 












.08 




Alg 1, Alg 2, Geom, & 
Other Adv Math 


.51 


.16 




















3 


Intercept 


-9.64 




.13 


.01* 


.13 


-.66 


.67 


.21* 


.38* 


-.60 


-.17 




PLAN Mathematics 


.50 


-.03 


>-.01 






.01* 


-.03 


-.01 


>-.01 








Grade level 


-.54 










.15 


-.12* 


.08* 


.03* 


-.14* 


.07 




Alg 1, Alg 2, Geom, & 
Trig 


.76 






.20 


.16 














4 


Intercept 


-9.44 




.15 






-.96 


.14* 


.08* 


.50* 


.24* 


-.14 




PLAN Mathematics 


.50 


-.03 


>-.01 






.03 


-.01* 


-.01* 


-.01* 


-.01* 






Grade level 


-.52 










.18 


-.03* 


.11 


.08* 


-.34 






Alg 1, Alg 2, Geom, Trig 
& Other Adv Math 


1.00 


.20 




















5 


Intercept 


-9.90 




.15 






-.40 


.37* 


.38 


.58* 


-.93 


-.17 




PLAN Mathematics 


.52 


-.03 


>-.01 






<.01* 


-.02* 


-.02 


-.01* 


.05 






Grade level 


-.50 






.04* 


.12 












.11 




Alg 1, Alg 2, Geom, Trig 
& Calculus 


1.53 






.27 


.13* 














6 


Intercept 


-9.89 




.15 






-.21* 


.12* 


.52 


.64* 


-1.07 






PLAN Mathematics 


.50 




>-.01 






-.01* 


-.01* 


-.03 


-.01* 


.06 






Grade level 


-.41 




















.06 




Alg 1, Alg 2, Geom, Other 
Adv Math, Trig, & Calc 


1.69 


-.22 




.27 


.19 


.20 


.17* 


.04* 


-.43 


.01* 





*p>.01 












ACT Science College Readiness Benchmark 



Model 


Level 1 


Level 2 regression coefficient 


Intercept 


Pub vs. 
priv. 


Class 

size 


Location 


Accred. region 


Multi- 

school 

dist. 


Rural vs. 
urban 


Suburban 
vs. urban 


South 


NW 


NC 


W 


NE 


1 


Intercept 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


PLAN Science 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


Grade level 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


Biology vs. General 
Science 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


** 


Biology vs. 


2 


Intercept 


-11.13 




.04 






.10* 


-.64 


.07* 


.71 


-.24* 


-.14 


PLAN Science 


.49 










-.01* 


.05 


-.01* 


-.02* 


>-.01* 




Grade level 


-.31 






.02* 


.09 














Biology & Chemistry 


.88 


-.22 








-.17 


-.01* 


.08* 


-.16* 


.27 




3 


Intercept 


-10.25 










-.06* 


-.74 


-.06* 


.73* 


.14* 




PLAN Science 


.45 










>-.01* 


.05 


>-.01* 


-.03* 


-.02* 




Grade level 


-.14 


-.19 


.02 
















-.16 


Biology, Chemistry, & 
Physics 


1.13 






.11 


.16 


-.15 


-.03* 


.18 


-.26* 


.27 





*p>.01 

** Insufficient numbers of students meeting the Science Benchmark to calculate. 






